Large graph convolutional network training with GPU-oriented data communication architecture

Graph Convolutional Networks (GCNs) are increasingly adopted in large-scale graph-based recommender systems. Training GCN requires the minibatch generator traversing graphs and sampling sparsely located neighboring nodes to obtain their features. Since real-world often exceed capacity of GPU memory, current training systems keep feature table host memory rely on CPU collect sparse features before sending them GPUs. This approach, however, puts tremendous pressure bandwidth CPU. is because needs (1) read from (2) write into as a dense format, (3) transfer In this work, we propose novel GPU-oriented data communication approach for training, where threads directly access through zero-copy accesses without much help. By removing gathering stage, our method significantly reduces consumption resources latency. We further present two important techniques achieve high efficiency by GPU: automatic address alignment maximize PCIe packet efficiency, asynchronous kernel execution fully overlap with training. incorporate PyTorch evaluate its effectiveness using several sizes up 111 million 1.6 billion edges. multi-GPU setup, 65--92% faster than conventional method, can even match performance all-in-GPU-memory some that fit memory.